Robust distribution of ip timing signals

ABSTRACT

Embodiments disclosed herein provide techniques to selectively distribute Precision Time Protocol (PTP) data in a network. The network can include multiple different network devices (e.g., switches) connected to form a network architecture (e.g., a spine/leaf architecture). Rather than distributing the PTP data (e.g., PTP timestamps) through all the network devices in order to synchronize local clocks to a global, master clock, the embodiments herein describe an out-of-band distribution network which selectively distributes the PTP data to select network devices in the network.

BACKGROUND Field of the Disclosure

Embodiments disclosed herein relate to selectively distributingPrecision Time Protocol (PTP) time reference data in network.

Description of the Related Art

The Precision Time Protocol (PTP) is defined by the Institute ofElectrical and Electronics Engineers (IEEE®) 1588 standard and adoptedby the Society of Motion Picture and Television Engineers® (SMPTE®)2059. The PTP standard provides a timestamp that has 80 bits ofprecision and is accurate to the microsecond range. Generally, SMPTE2059 is directed to leveraging PTP to provide timestamps for video datathat is encapsulated in Internet Protocol (IP) packets. Using theseprotocols, PTP time reference data can be shared throughout a network toprovide a synchronized clock to media nodes (e.g., cameras, audiodevices, video encoders, audio encoders, etc.) connected to the network.That is, the switches and other network devices distribute clock signals(e.g., timestamps) that can synchronize local clocks to a master clock.

However, every hop in the network introduces inaccuracy into the PTPsignals. That is, even when the switches in the network are PTP-enabled,the switches introduce jitter (e.g., 5-20 nanoseconds) which can reducethe accuracy between a local clock and the master clock. Further, if apath in the network traverses a non-PTP-enabled switch, the jitter orinaccuracy in the PTP signals is increased more, since the non-PTPswitch does not account for the transit time. That is, the receivingswitches do not know the duration of the transit delay through thenon-PTP switch. While this inaccuracy can be mitigated using onlyPTP-enabled switches, these switches are more expensive. Further, PTPrequires frequent communication between the network devices, which canuse significant portions of the bandwidth in the network. As networksgrow larger, distributing PTP signals throughout the network can beexpensive and reduce the available bandwidth in that network. Furtherlarger networks suffer from less accurate clocks signals due to theadditive effect of jitter at each of the network switches andinaccuracies discussed above.

SUMMARY

In one embodiment, a network that includes a plurality of PTP-enablednetwork devices communicatively coupled to a plurality of media nodes, aplurality of non-PTP enabled network devices communicatively coupled tothe plurality of PTP-enabled network devices, and an out-of-band PTPdistribution network communicatively coupled to the plurality ofPTP-enabled network devices. The out-of-band distribution networkincludes a master clock device configured to generate a master clock anda transparent clock switch where the transparent clock switch iscommunicatively coupled between the master clock device and theplurality of PTP-enabled network devices. Moreover, the transparentclock switch measures a transit delay, and inserts the transit delay,into PTP packets exchanged between the master clock device and theplurality of PTP-enabled network devices. Further, the plurality ofPTP-enabled network devices is configured to transmit only non-PTPsignals to the plurality of non-PTP enabled network devices

In another embodiment, a method for synchronizing boundary clocks in aplurality of PTP-enabled network devices is described. The methodincludes receiving, at the plurality PTP-enabled network devices, anannouncement from a master clock device were the announcement isreceived via a transparent clock switch and where the master clockdevice and the transparent clock switch form an out-of-band PTPdistribution network. The method also includes synchronizing theboundary clocks to a master clock generated by the master clock devicebased on a timestamp of the master clock and a transit delaycorresponding to the transparent clock switch, transmitting timestampsof the boundary clocks to media nodes connected to the plurality ofPTP-enabled network devices, and transmitting non-PTP data signals fromthe plurality of PTP-enabled network devices to a plurality of non-PTPenabled network devices.

In still another embodiment, an out-of-band distribution networkincludes a PTP-enabled network device, a master clock device configuredto generate a master clock, and a transparent clock switchcommunicatively coupled between the master clock device and thePTP-enabled network device. The transparent clock switch measures atransit delay, and inserts the transit delay, into PTP packets exchangedbetween the master clock device and the PTP-enabled network device.Further, the PTP-enabled network device is communicatively coupled to anon-PTP enabled network device.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited aspects are attained andcan be understood in detail, a more particular description ofembodiments of the disclosure, briefly summarized above, may be had byreference to the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this disclosure and are therefore not to beconsidered limiting of its scope, for the disclosure may admit to otherequally effective embodiments.

FIG. 1 illustrates selectively distributing PTP signals using anout-of-band distribution network, according to one embodiment.

FIG. 2 illustrates an out-of-band distribution network, according to oneembodiment.

FIG. 3 is a flowchart for synchronizing an out-of-band distributionnetwork with a master clock, according to one embodiment.

FIG. 4 is a flowchart for recovering from lost synchronization in anout-of-band distribution network, according to one embodiment.

FIG. 5 illustrates multiple sites with respective out-of-banddistribution networks, according to one embodiment.

DETAILED DESCRIPTION

Embodiments disclosed herein provide techniques to selectivelydistribute Precision Time Protocol (PTP) data in a network. The networkcan include multiple different network devices (e.g., switches)connected to form a network architecture (e.g., a spine/leafarchitecture). Rather than distributing the PTP data (e.g., PTPtimestamps) through all the network devices in order to synchronizelocal clocks to a global, master clock, the embodiments herein describean out-of-band distribution network that selectively distributes the PTPdata to select network devices in the network(i.e., a subset of networkdevices less than a total of all network devices in the network). Thatis, some of the network devices receive PTP data (which they can use tosynchronize local clocks to a master clock) while other network devicesdo not. The devices connected to the network that rely on thePTP-synchronized clocks (referred to generally as media nodes) arecoupled to the network devices coupled to the out-of-band distributionnetwork. Doing so reduces the cost of the switches (since not all of theswitches need to be PTP enabled) and increases the accuracy of the PTPtimestamps since the PTP packets reach the network devices via theout-of-band distribution network which may have less hops than datapaths in a traditional network. Further, enabling a select number of thenetwork devices increases security since doing so provides less accesspoints for an unintended actor (e.g., a nefarious actor, an accidentalactor, a rogue actor, etc.) to couple an unapproved device, which thenetwork devices may incorrectly determine is a master clock (and maysynch to that clock rather than an authorized master clock).

In one embodiment, the out-of-band distribution network includesmultiple master clock devices which independently generate masterclocks. The master clock devices may be coupled to transparent clockswitches which distribute the master clocks to the network devices inthe network selected to receive the PTP signals (referred to herein asboundary clocks). The transparent clock switches are pass throughdevices that track the transit delays of PTP packets being transferredfrom the master clock devices to the boundary clocks and vice versa. Theout-of-band distribution network can have multiple master clock devicesand multiple transparent clock switches to provide redundancy andresiliency. If one of the master clock devices or transparent clockswitches fails or is unavailable due to an upgrade or other reason, theboundary clocks can synchronize to another one of the master clocks, orselect the master clock signal via a different path through a differentone of the transparent clock switches. In one embodiment, the PTPsignals are permitted to travel only within the out-of-band distributionnetwork, the network devices with the boundary clocks, and the medianodes coupled to the boundary clocks. For example, the network deviceswith the boundary clocks may not transmit PTP data to other networkdevices in the network. Thus, the PTP signals are limited to theselected network devices (which are PTP enabled) and the out-of-banddistribution network, which tightly controls the latency and jitter ofthe PTP signals and minimizes security risk.

FIG. 1 illustrates selectively distributing PTP signals using anout-of-band distribution network 150, according to one embodiment. FIG.1 includes a network 100 with a plurality of network devices—i.e., spineswitches 105, non-PTP leaf switches 110 (referred to generally asnon-PTP enabled network devices), and PTP-leaf switches 115 (referred togenerally as PTP enabled network devices). These network devices provideconnectivity to client devices 125 and media nodes 130. The media nodes130 may be any computing device that uses PTP reference signals todecode or generate IP-based media signals. Some non-limiting examples ofmedia nodes 130 includes cameras, audio recording devices, mixers, videoand audio encoders, decoders, and the like. The client devices 125, onthe other hand, may be any computing devices that do not rely on theaccurate clock signals which PTP provides. Non-limiting examples ofclient devices 125 include personal computing devices (e.g., laptops orsmartphones), servers (e.g., a database server or storage server), orgeneral compute platforms. The spine switches 105 and the leaf switches110, 115 provide connectivity between the client devices 125 and themedia nodes 130 as well as connectivity to other networks (not shown).Further, while the PTP leaf switches 115 are illustrated being coupledto only the media nodes 130, the switches 115 can also be coupled to theclient devices 125.

The network 100 in FIG. 1 is arranged in a spine-leaf architecture whereevery spine switch 105 is coupled to each one of the leaf switches 110,115. While the leaf switches 110, 115 can have direct connections toeach other (which may be useful for backup, redundancy, and failoverpurposes), generally the leaf switches 110, 115 rely on forwarding datato a spine switch 105 which in turn forwards the data to another one ofthe leaf switches 110, 115 that is connected to the destination clientdevice 125 or media node 130. Although not shown, the spine switches 105(or a leaf switch 110, 115) may include connections (e.g., a virtuallocal access network (VLAN) connection) to a different site that has itsown network devices arranged in a spine-leaf architecture.

In FIG. 1, the PTP signals are provided by the out-of-band PTPdistribution network 150. However, the out-of-band PTP distributionnetwork 150 is not connected to every one of the switches, but rather isonly connected to select ones of the switches—i.e., the PTP leafswitches 115. The out-of-band PTP distribution network 150 is notconnected to the non-PTP leaf switches 110. In one embodiment, thenon-PTP leaf switches 110 may not be PTP aware. That is, if a non-PTPleaf switch 110 receives a PTP packet, it will forward the packet, butdoes not track a transit delay associated with processing the packet. Inother words, the non-PTP switch 110 treats the PTP packet the same asother data traffic. As a result, downstream network devices do not knowthe transit delay which reduces the accuracy of the timestamps in theforwarded PTP data. Often, non-PTP leaf switches 110 are less expensivethan the PTP leaf switches 115. As a result, by selectively distributingPTP signals (via the out-of-band PTP distribution network 150) to thePTP leaf switches 115 (and not the non-PTP leaf switches 110), theoverall cost of the network 100 may be reduced.

In an alternative embodiment, rather than selectively distributing thePTP signals, all the switches (both spine and leaf) may be PTP enablednetwork devices. In that case, a master clock device is coupled to oneof the network devices (either a spine or leaf) and the PTP data can bedistributed to all of the network devices. However, this may drive upthe cost of network. Further, the accuracy of the PTP data may bereduced relative to the network 100 in FIG. 1. For example, the numberof hops between network devices may vary, where each hop adds additionaljitter (even when the network devices are all PTP enabled). Thus, medianodes may be plugged into network devices with more hops between a medianode and the master clock, thereby reducing the accuracy of thecorresponding timestamps. Further, if all the network devices are PTPenabled, someone (either unknowingly or intentionally) may connect anunapproved master clock into one of the network devices. The PTP networkdevices may synchronize to the unapproved master clock rather than theapproved master clock coupled elsewhere to the network. The PTPprotocol, as of now, does not include techniques for authorizing masterclocks. For example, while the Best Master Clock Algorithm (BMCA) has amethod of authorizing a master clock, currently there is no securityassociated with this algorithm, so there is no method for secureauthorization of a master clock

By using the out-of-band PTP distribution network 150, these concernscan be mitigated. The overall cost of the network 100 can be reduced,any jitter is tightly controlled, and any security concerns aremitigated due to the fact that the network 100 includes limited PTP leafswitches 115, which makes controlling access to these switches easier.

While FIG. 1 illustrates a spine-leaf architecture, the embodimentsherein are not limited to such. The network 100 can include any type ofnetwork architecture for coupling together network devices at a site orcampus. Put differently, regardless of the specific interconnectivity ofthe network devices in a network, the out-of-band PTP distributionnetwork 150 can be used to selectively distribute PTP signals to certainnetwork devices (i.e., a subset of network devices less than a total ofall network devices in the network). These devices, in turn, can becoupled to media nodes 130 which rely on the PTP data to perform theirfunctions.

FIG. 2 illustrates the out-of-band PTP distribution network 150,according to one embodiment. The out-of-band PTP distribution network150 includes a plurality of master clock devices 205 and a plurality oftransparent clock switches 225. For context, FIG. 2 also illustrates theplurality of PTP leaf switches 115 which, in one embodiment, may beconsidered as part of the out-of-band PTP distribution network 150, aswell as part of the data network.

The master clock devices 205 include respective master clocks 215 and aglobal positioning system (GPS) receiver 210. The master clock devices205 may be specialized devices for generating PTP data which includestimestamps corresponding to respective master clocks 215. In oneembodiment, the master clock devices 205 are appliances designed for PTPapplications. The receiver 210 is configured to receive GPS signalswhich have timestamps (or clock information). The master clock device205 is configured to then synchronize its master clock 215 to the clockcorresponding to GPS. While FIG. 2 illustrates multiple master clockdevices 205 in the out-of-band PTP distribution network 150, this is nota requirement. Although one master clock device 205 is sufficient,providing additional master clock devices 205 provides redundancy if onefails, is upgraded, or is otherwise unavailable.

The master clock devices 205 are coupled to transparent clock switches225. In one embodiment, the transparent clock switches 225 are networkdevices (while the master clock devices 205 may be an appliance ratherthan a network device). The transparent clock switches 225 are PTPenabled network devices which distribute the PTP signals generated bythe master clock devices 205 to the PTP leaf switches 115. In oneembodiment, the transparent clock switches 225 measure a transit delay230 which is the time it takes for PTP packets to traverse thetransparent clock switch 225. This delay 230 may be tracked for packetsthat are transmitted from the master clock devices 205 to the PTP leafswitches 115 and for packets transmitted from the PTP leaf switches 115to the master clock devices 205. Before forwarding a PTP packet, thetransparent clock switch 225 inserts the transit delay 230 into thepacket. That way, downstream devices (e.g., the master clock devices 205or the PTP leaf switches 115) know the delay 230 incurred when thepacket traversed the switch 225. This delay 230 can then be used to moreaccurately synchronize the boundary clocks 120 in the PTP leaf switches115 to a master clock 215.

Due to hardware constraints, each of the master clock devices 205 may beconfigured to communicate with up to two network devices. Thetransparent clock switches 225 receives the PTP signals from the masterclock devices 205 and then distributes the PTP signals to multiplenetwork devices. In FIG. 2, the master clock devices 205 are coupled toboth of the transparent clock switches 225. Thus, if one of thetransparent clock switches 225 fails, is being upgraded, or is otherwiseunavailable, the PTP data can still reach the master clock devices 205using the other transparent clock switch 225. Thus, while twotransparent clock switches 225 are not necessary, having multipletransparent clock switches 225 provides redundancy and failover in theout-of-band PTP distribution network 150. Further, while two transparentclock switches 225 are shown, the out-of-band PTP distribution network150 can have any number of transparent clock switches assuming thehardware in the master clock devices 205 supports connecting to morenetwork devices.

In one embodiment, the master clock devices 205 do not support Layer 3(L3) communication. As a result, the out-of-band PTP distributionnetwork 150 includes Layer 2 (L2) connections 220 between the masterclock devices 205 and the transparent clock switches 225. However,because the transparent clock switches 225 are network devices, thetransparent clock switches 225 can support L3 connections 235 to the PTPleaf switches 115. Using L3 connections 235 permit the transparent clockswitches 225 and the PTP leaf switches 115 to control the destination ofthe PTP signals. In one embodiment, the PTP data is routed as L3 routedmulticast data rather than a L2 connection because the PTP data reachesthe PTP leaf switches 115 outside of the normal leaf/spine architectureof the network 100 illustrated in FIG. 1. Doing so enables the PTP leafswitches 115 to ensure the PTP signals are routed to the transparentclock switches 225 rather than to other spine or leaf switches in thenetwork. That is, the L3 connections ensure the PTP leaf switches 115route the PTP data to the transparent clock switches 225 rather than thenon-PTP enabled connections 140 to spine or leaf switches that are notPTP enabled. However, the PTP leaf switches 115 can use the non-PTPenabled connections 140 to send/receive non-PTP data to/from the othernetwork devices in the network. Thus, the PTP data can be isolated fromthe remaining network, or stated differently, the PTP data is limited tothe out-of-band PTP distribution network 150 (which can include the PTPleaf switches 115 and the media nodes). Further, the L3 connections 235may be used for transmitting only PTP data, but not non-PTP data, whichis transmitted using the non-PTP enabled connections 140 and thePTP-enabled connections 135. In one embodiment, the PTP multicast and/orunicast address is blocked from the leaf/spine data network 100, even ifmulticast is enabled on the network 100.

The PTP leaf switches 115 forward the PTP data to the media nodes (whichare connected to the switches 115) using the PTP-enabled connections135. These PTP-enabled connections 135 may be L2 connections since manymedia nodes do not support L3 communication. Moreover, the PTP leafswitches 115 can also use the PTP-enabled connections 135 to exchangenon-PTP data packets with the media nodes. That is, the PTP-enabledconnections 135 can transmit both PTP data packets and normal datapackets between the PTP leaf switches 115 and the media nodes.

The PTP leaf switches 115 use the PTP data exchanged with the masterclock devices 205 to synchronize respective boundary clocks 120 to oneof the master clocks 215 in a process that is described in more detailbelow. In one embodiment, the boundary clocks 120 are servant or slaveclocks to the selected master clock 215. The boundary clock 120 may be“syntonized” to the master clock 215. Syntonized is the term used in theIEEE-1588 standard to describe a clock that is held closely in time toanother master clock. Synchronized is not used in the IEEE standard asit implies a fully synchronous lock, which is not technically possiblein a system without continuous loop feedback between clocks—such as apacket-switched network. In any case, the boundary clocks 120 can besyntonized or synchronized to the master clock 215. Thus, the PTP leafswitches 115 can transmit timestamps generated by their respectiveboundary clocks 120 to the media nodes which the media nodes can usewhen performing their IP-based media functions. That is, rather thanreceiving timestamps directly from the master clock devices 205, themedia nodes rely on timestamps generated from the boundary clocks 120 toperform their functions.

FIG. 3 is a flowchart of a method 300 for synchronizing an out-of-banddistribution network with a master clock, according to one embodiment.For ease of explanation, the method 300 is discussed in the context ofthe out-of-band distribution network 150 illustrated in FIG. 2.

At block 305, the PTP leaf switches receive announcements from themaster clock devices. As shown in FIG. 2, these announcements maytraverse the transparent clock switches. However, rather thansynchronizing their local clocks to the master clock, the transparentclock switches instead identify the transit delay associated with thePTP packets and insert this delay into the packets before forwarding thepackets. The transparent clock switches can insert this delay for all(or at least a subset) of the packets transmitted through it.

If the out-of-band PTP distribution network includes multiplefunctioning master clock devices, the PTP leaf switches may receiveannouncement messages from all the master clock devices. However, withPTP, only one of the master clocks and its corresponding master clockdevices is selected as the master clock to which the PTP leaf switchessynchronize their boundary clocks.

At block 310, the PTP leaf switches select one of the master clockdevices to use when synchronizing their boundary clocks. In oneembodiment, the PTP leaf switches use the BMCA to identify the optimalmaster clock (e.g., a grand master clock) to use for syntonization orsynchronization. Regardless of the particular algorithm used, if the PTPleaf switches use the same algorithm, they select the master clockgenerated by the same master clock device. If no grand master clocks areavailable, the PTP leaf switches may be preconfigured such that the PTPleaf switches select a predetermined PTP leaf switch as the master clocksource.

At block 315, the PTP leaf switches synchronize to the selected masterclock device, while accounting for the transit delay introduced by thetransparent clock switch used to route data between the PTP leafswitches and the master clock devices. The particular details forsynchronizing the boundary clocks to the selected master clock are notprovided herein. However, as a brief overview, the PTP leaf switches andthe master clock devices may exchange multiple PTP packets that includetimestamps generated by the selected master clock and the boundaryclocks. Using these timestamps, the PTP leaf switches can determine anoffset between the boundary clocks and the selected master clock. Thetransit delay through the transparent clock switch can also be used whencalculating the offset. The offset determined by each PTP leaf switch isthen used to synchronize its boundary clock to the selected masterclock.

The PTP leaf switches can continue to exchange synchronization data withthe selected master device to maintain synchronization. The PTP leafswitches can adjust their boundary clocks (e.g., the servant clocks) tomaintain synchronization with the master clock. As mentioned above,these periodic messages can use a significant amount of bandwidth.However, because the PTP traffic is limited or restricted to theout-of-band PTP distribution network, the PTP traffic does not usebandwidth in the portion of the network used to transmit data betweenthe switches. Put differently, the PTP traffic does not use thebandwidth used by other data traffic.

At block 320, the PTP leaf switches transmit the synchronized boundaryclock to connected media nodes. That is, now that the boundary clocks inthe PTP leaf switches are synchronized to the selected master clock, themedia nodes can receive accurate timestamps of the boundary clock whichthey can use to perform their intended functions (e.g., capturing orediting audio or visual information in a media presentation). Further,the media nodes can rely on timestamps of the boundary clock rather thancommunicating with the master clock (which may not be able toeffectively communicate with a plurality of media nodes).

At block 325, the PTP leaf switches transmit data signals to othernetwork devices in the network using non-PTP enabled connections. Thatis, in FIG. 2, the PTP leaf switches 115 have both PTP-enabledconnections 135, which transmit both normal data traffic and PTP trafficto the media nodes, as well as non-PTP connections 140, which are usedto transmit non-PTP data (e.g., all other data traffic besides PTP) tothe spine and leaf switches in the network.

FIG. 4 is a flowchart of a method 400 for recovering from lostsynchronization in an out-of-band distribution network, according to oneembodiment. At block 405, a PTP leaf switch detects a loss ofsynchronization with the selected master clock device. This may occurwhen a master clock device fails or is being upgraded. Further, a lossof synchronization may occur when a transparent clock switch fails or isbeing upgraded.

At block 410, the out-of-band distribution network determines whether analternative path to the master clock is available. For example, theout-of-band PTP distribution network may include multiple transparentclock switches that provide redundant, failover paths to the masterclock devices. If one of the transparent clock switches fails, the datatraffic can automatically failover to another transparent clock switch,which provides an alternative path.

Assuming an alternative path is available, at block 415, the PTP leafswitch (or switches) use the alternative path to re-establishsynchronization with the selected master clock device. For example, thePTP packets can automatically begin using the redundant transparentclock switch to exchange synch data with the selected master device.

However, if the selected master device fails or is otherwiseunavailable, there may not be an alternative path. In that case, themethod 400 proceeds to block 420 where the PTP leaf switches determinewhether another master clock device is available. For example, theout-of-band PTP distribution network may include multiple master clockdevices which have independent master clocks. If the selected masterclock fails, the PTP leaf switches can synchronize with one of the othermaster clock devices. In that case, the method 400 proceeds to block 425where the PTP leaf switches select one of the available master clockdevices. The PTP leaf switches can use the same techniques describedabove in FIG. 3 to select a master clock—e.g., BMCA. The method 400 canthen proceed to block 315 of method 300 to synchronize the PTP leafswitches to the newly selected master clock device.

However, if there are no available master clocks (e.g., the out-of-bandPTP distribution network may have only one master clock, or all themaster clocks devices may have failed), the method 400 proceeds to block430 where the PTP leaf switches select a boundary clock as the masterclock. In this example, the boundary clock in one of the PTP leafswitches is selected as the master clock to which the other PTP leafswitches synchronize. That way, all of the media nodes can continue toreceive timestamps from the same master clock, which they can use toperform their respective functions.

FIG. 5 illustrates multiple sites with respective out-of-banddistribution networks, according to one embodiment. In one embodiment,the three sites 505A-505C illustrated in FIG. 5 can form a singlenetwork that is communicatively coupled by inter-site connections510A-510C. For example, the sites 505A-505C may form a private networkby which the sites 505A-505C can communicate without traversing a publicnetwork. To do so, the inter-site connections 510A-510C may be virtuallocal access network (VLAN) connections that encrypt or segregate thedata traffic exchanged by the sites 505A-505C. In one embodiment, thesites 505A-505C and the inter-site connections 510A-510C may form theprivate network for an entity such as a company that has local resources(e.g., the switches 105, 110, and 115) at different geographic locations(e.g., Los Angeles, New York, and London). However, in anotherembodiment, the sites 505A-505C may be different networks that rely on apublic network (e.g., the Internet) to exchange data.

As shown, each site 505A, 505B, 505C has its own respective out-of-banddistribution network 150A, 150B, 150C. While it may be possible to haveone out-of-band distribution network and distribute the PTP signals aplurality of sites, as discussed above, this adds jitter andinaccuracies, and also may require that all the switches are PTP enabledfor achieving accurate PTP synchronization. Instead, in FIG. 5, eachsite 505A, 505B, 505C has its own respective out-of-band PTPdistribution network 150A, 150B, 150C that selectively exchanges PTPsynch data only with the PTP leaf switches 115, rather than all the leafswitches in the sites 505A, 505B, 505C. Doing so ensures the media nodesat each site 505A-505C have more accurate PTP signals (relative todistributing the PTP signals using the inter-site connections510A-510C), can reduce overall costs since not all of the leaf switchesand spine switches need to be PTP enables, and can improve security. Inthis example, the out-of-band PTP distribution network 150A provides PTPsignals only for the PTP leaf switches 115 at the site 505A, theout-of-band PTP distribution network 150B provides PTP signals only forthe PTP leaf switches 115 at the site 505B, and the out-of-band PTPdistribution network 150C provides PTP signals only for the PTP leafswitches 115 at the site 505C.

In the current disclosure, reference is made to various embodiments.However, it should be understood that the present disclosure is notlimited to specific described embodiments. Instead, any combination ofthe preceding features and elements, whether related to differentembodiments or not, is contemplated to implement and practice theteachings provided herein. Additionally, when elements of theembodiments are described in the form of “at least one of A and B,” itwill be understood that embodiments including element A exclusively,including element B exclusively, and including element A and B are eachcontemplated. Furthermore, although some embodiments may achieveadvantages over other possible solutions or over the prior art, whetheror not a particular advantage is achieved by a given embodiment is notlimiting of the present disclosure. Thus, the aspects, features,embodiments and advantages disclosed herein are merely illustrative andare not considered elements or limitations of the appended claims exceptwhere explicitly recited in a claim(s).

As will be appreciated by one skilled in the art, embodiments describedherein may be embodied as a system, method or computer program product.Accordingly, embodiments may take the form of an entirely hardwareembodiment, an entirely software embodiment (including firmware,resident software, micro-code, etc.) or an embodiment combining softwareand hardware aspects that may all generally be referred to herein as a“circuit,” “module” or “system.” Furthermore, embodiments describedherein may take the form of a computer program product embodied in oneor more computer readable medium(s) having computer readable programcode embodied thereon.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for embodiments of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present disclosure are described herein with reference toflowchart illustrations or block diagrams of methods, apparatuses(systems), and computer program products according to embodiments of thepresent disclosure. It will be understood that each block of theflowchart illustrations or block diagrams, and combinations of blocks inthe flowchart illustrations or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe block(s) of the flowchart illustrations or block diagrams.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other device to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the block(s) of the flowchartillustrations or block diagrams.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other device to cause aseries of operational steps to be performed on the computer, otherprogrammable apparatus or other device to produce a computer implementedprocess such that the instructions which execute on the computer, otherprogrammable data processing apparatus, or other device provideprocesses for implementing the functions/acts specified in the block(s)of the flowchart illustrations or block diagrams.

The flowchart illustrations and block diagrams in the Figures illustratethe architecture, functionality, and operation of possibleimplementations of systems, methods, and computer program productsaccording to various embodiments of the present disclosure. In thisregard, each block in the flowchart illustrations or block diagrams mayrepresent a module, segment, or portion of code, which comprises one ormore executable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order or out of order, dependingupon the functionality involved. It will also be noted that each blockof the block diagrams or flowchart illustrations, and combinations ofblocks in the block diagrams or flowchart illustrations, can beimplemented by special purpose hardware-based systems that perform thespecified functions or acts, or combinations of special purpose hardwareand computer instructions.

While the foregoing is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A network, comprising: a plurality of PrecisionTime Protocol (PTP)-enabled network devices communicatively coupled to aplurality of media nodes; a plurality of non-PTP enabled network devicescommunicatively coupled to the plurality of PTP-enabled network devices;an out-of-band PTP distribution network communicatively coupled to theplurality of PTP-enabled network devices, the out-of-band PTPdistribution network comprising: a master clock device configured togenerate a master clock, and a transparent clock switch, wherein thetransparent clock switch is communicatively coupled between the masterclock device and the plurality of PTP-enabled network devices, whereinthe transparent clock switch measures a transit delay, and inserts thetransit delay, into PTP packets exchanged between the master clockdevice and the plurality of PTP-enabled network devices, wherein theplurality of PTP-enabled network devices is configured to transmit onlynon-PTP signals to the plurality of non-PTP enabled network devices. 2.The network of claim 1, wherein the out-of-band PTP distribution networkcomprises: a plurality of master clock devices, each generating anindependent master clock; and a plurality of transparent clock switches,wherein each of the plurality of transparent clock switches iscommunicatively coupled to each of the plurality of master clock devicesand each of the plurality of PTP-enabled network devices, wherein theplurality of transparent clock switches serves as a backup to eachother.
 3. The network of claim 1, further comprising: a plurality ofspine switches, wherein the pluralities of PTP- and non-PTP-enablednetwork devices are leaf switches, wherein the plurality of spineswitches is non-PTP enabled.
 4. The network of claim 3, wherein theplurality of spine switches exchange only non-PTP packets with theplurality of PTP-enabled network devices.
 5. The network of claim 3,further comprising a plurality of PTP connections and a plurality ofnon-PTP connections, wherein the plurality of PTP-enabled networkdevices are configured to use the PTP connections to communicate withthe transparent clock switch and the media nodes, and to use the non-PTPconnections to communicate with the plurality of non-PTP-enabled networkdevices and the plurality of spine switches.
 6. The network of claim 1,wherein the plurality of PTP-enabled network devices use at least Layer3 routing to transmit the PTP packets to the transparent clock.
 7. Thenetwork of claim 1, wherein the master clock device comprises a globalpositioning system (GPS) receiver for generating the master clock andthe plurality of PTP-enabled network devices comprise respectiveboundary clocks configured to synchronize to the master clock using thePTP packets.
 8. A method for synchronizing boundary clocks in aplurality of PTP-enabled network devices, comprising: receiving, at theplurality PTP-enabled network devices, an announcement from a masterclock device, wherein the announcement is received via a transparentclock switch, wherein the master clock device and the transparent clockswitch form an out-of-band PTP distribution network; synchronizing theboundary clocks to a master clock generated by the master clock devicebased on a timestamp of the master clock and a transit delaycorresponding to the transparent clock switch; transmitting timestampsof the boundary clocks to media nodes connected to the plurality ofPTP-enabled network devices; and transmitting non-PTP data signals fromthe plurality of PTP-enabled network devices to a plurality of non-PTPenabled network devices.
 9. The method of claim 8, further comprising:receiving, at the plurality of PTP-enabled network devices, a secondannouncement message from a second master clock device in theout-of-band PTP distribution network, wherein the transparent clockswitch is connected between the second master clock device and theplurality of PTP-enabled network devices; and selecting between themaster clock device and the second master clock device based on theannouncement message and the second announcement message.
 10. The methodof claim 9, wherein a second transparent clock switch is coupled to eachof the plurality of PTP-enabled network devices and both of the masterclock device and the second master clock device.
 11. The method of claim10, wherein the transparent clock switch and the second transparentclock switch serve as backups to each other.
 12. The method of claim 10,further comprising: detecting a loss of synchronization with the masterclock device; and synchronizing the boundary clocks to the second masterclock device.
 13. The method of claim 8, further comprising:transmitting non-PTP data from the plurality of PTP-enabled networkdevices to a plurality of spine switches which is not PTP enabled,wherein the pluralities of PTP- and non-PTP-enabled network devices areleaf switches.
 14. The method of claim 8, further comprising: routingsynchronization packets between the plurality of PTP-enabled networkdevices and the transparent clock switch using at least a Layer 3routing protocol.
 15. An out-of-band distribution network, comprising: aPTP-enabled network device; a master clock device configured to generatea master clock, and a transparent clock switch communicatively coupledbetween the master clock device and the PTP-enabled network device,wherein the transparent clock switch measures a transit delay, andinserts the transit delay, into PTP packets exchanged between the masterclock device and the PTP-enabled network device, wherein the PTP-enablednetwork device is communicatively coupled to a non-PTP enabled networkdevice.
 16. The out-of-band distribution network of claim 15, furthercomprising: a plurality of master clock devices, each generating anindependent master clock; and a plurality of transparent clock switches,wherein each of the plurality of transparent clock switches iscommunicatively coupled to each of the plurality of master clock devicesand the PTP-enabled network device, wherein the plurality of transparentclock switches serves as backups to each other.
 17. The out-of-banddistribution network of claim 15, wherein the PTP-enabled network deviceis communicatively coupled to a plurality of spine switches, wherein thePTP-enabled network device is leaf switch, wherein the plurality ofspine switches is non-PTP enabled.
 18. The out-of-band distributionnetwork of claim 17, wherein the plurality of spine switches exchangeonly non-PTP packets with the PTP-enabled network device.
 19. Theout-of-band distribution network of claim 17, wherein the PTP-enablednetwork device uses a plurality of PTP connections to communicate withthe transparent clock switch and media nodes but uses non-PTPconnections to communicate with the non-PTP-enabled network device andthe plurality of spine switches.
 20. The out-of-band distributionnetwork of claim 15, wherein the PTP-enabled network device uses atleast Layer 3 routing to transmit the PTP packets to the transparentclock switch.